Motion encoding of still images

ABSTRACT

The present invention is directed to a method and apparatus for encoding a series of still images. The first image is captured and stored, then for each subsequent image of the series, that image is captured and compared to the first image or the previous image and difference information between the first or previous image and each subsequent image of the series is stored, resulting in one output file that contains the base image and difference information for each subsequent image, thereby reducing the amount of storage needed to store a series of images.

FIELD OF THE INVENTION

The present invention generally relates to the capture of images, andmore particularly relates to compressing and storing digital stillphotographic images in a more efficient manner.

BACKGROUND OF THE INVENTION

Digital cameras capture and store a large number of digital images.Fortunately, the capacity of memory cards used to store these images isincreasing so that the cards are able to hold larger and larger numbersof images. Working against this increasing capacity in memory cards isthe increasing resolution of image sensors in the cameras, which requirethe creation of larger files to store each individual image. The totalnumber of images that can be stored on a particular memory card isproportional to the size of the memory card, the resolution of theimages, and the effectiveness of the compression algorithms used toreduce the storage needed for each image. Some images are notcompressed, while others are compressed with algorithms such as used inthe JPEG standard, yielding smaller image files. Certain features ofsome more advanced digital cameras make it easy to fill up a memory cardwith large numbers of images. Two features, in particular, create arapid succession of images in a short period of time. One of thefeatures is bracketing, where a succession of several images iscaptured, each with a slight variation of certain camera settings. Theother of the features is a repetition mode, in which the user can take asequence of images, possibly by holding the shutter down. The formerfeature is useful when it is hard to set the camera for a certainexposure. The latter feature is useful for taking pictures of an objectthat is changing, such as a child that may not be smiling when the firstpicture is taken, but may subsequently smile when the second, third orfourth pictures are taken. There are other features that may alsoproduce a series of similar images. In both instances, there is a highprobability that certain aspects of each captured image will be similaror the same. For example, the backgrounds may all be similar while thesubject may change in each frame.

Having these features may also create another problem, in that, takingso many images may quickly fill the user's memory card with sets ofsimilar images. Still image compression, such as JPEG, may reduce theoverall size of each individual image, but, at perhaps a megabyte perimage, taking several sets of 4 to 5 images in rapid succession mayquickly fill up even the highest capacity memory card. Another problemis the number of images that fit within the camera's higher-speedtemporary memory. When taking the succession of images, currenttechnology stores each image of the succession in a higher-speedtemporary memory before moving to longer term storage, perhaps becausethere isn't sufficient time to store each image on a memory card. Sincethe higher-speed temporary memory is limited in size, only a smallnumber of successive images may be stored within such memory. If abetter compression method was available, a greater number of successiveimages could be captured.

Therefore, it would be desirable for a multi-image compression mechanismthat will recognize a series of images and use motion encodingcompression schemes to further compress the series of images.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a system and methodfor compressing still images using motion encoding.

In one aspect of the present invention, a series of images is analyzedas if the images were frames of a motion picture, using the first imageas a base image and each subsequent image as an offset to that baseimage, so that each subsequent image is actually encoded as informationabout any changes, or delta information, from the base image, therebyreducing the overall storage required for the series of images.

In another aspect of the present invention, as a series of images arecaptured, the first image is used as a base image and each subsequentimage is encoded as delta information with respect to the image beforeit. One single file may be created on the storage medium containing thebase image and each subsequent delta image. Each image may be viewed byopening the base image and applying each set of delta information to thebase image until arriving at the desired image. Furthermore, once theimage set is copied onto a system with greater storage capacity, such asa personal computer, the set can be divided into atomic or completeimages, perhaps storing each image of the series in a separate file,perhaps in a JPEG format, for each image.

In another aspect of the present invention, a camera captures a seriesof related images, then combines each image of the series into one fileby using the first image as a base image and each subsequent image isrecorded as delta information from the previous image in a mannersimilar to MPEG-2 encoding.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention as claimed. The accompanyingdrawings, which are incorporated in and constitute a part of thespecification, illustrate an embodiment of the invention and togetherwith the general description serve to explain the principles of theinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The numerous advantages of the present invention may be betterunderstood by those skilled in the art by reference to the accompanyingfigures in which:

FIG. 1 is a schematic block diagram of an illustrative systemimplementing the present invention.

FIG. 2 is a flow chart of a first aspect of the present invention.

FIG. 3 is a flow chart of a second aspect of the present invention.

FIG. 4 is a representation of a set of images that may be compressedusing the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the presently preferredembodiments of the invention, examples of which are illustrated in theaccompanying drawings.

Referring generally now to FIG. 1, a schematic representation of anelectronic system according to the present invention is shown. Thesystem may be an electronic device 100, such as, for example, a digitalphotographic camera. The system includes a relatively longer termstorage element for storing images, such as a memory card 110; arelatively shorter term storage element for storing program informationas well as temporary images during the capture process, such astemporary memory 115; an image processor 120 for controlling the systemand processing captured images; an image sensor 130 for capturing theimage; a keypad 150 for accepting user input and commands; and a display140 for displaying menus, images and perhaps a viewfinder image. Itshould be noted that this is an illustrative example of a system of thepresent invention, such as a camera, and that the present invention isnot limited to this illustrative system, nor is the method of thepresent invention restricted to being performed only within a camerasystem. Furthermore, the present invention can also be practiced by acomputer system, and may be performed by the computer system on a seriesof images that are already stored on the hard drive of the computersystem.

In the present invention, inputs on the keypad 150 that are guided bymenus on the display 140 may guide the user to set the camera to takemultiple images, perhaps in rapid succession. Prior to the presentinvention, the image processor 120 would extract digital images atperiodic intervals from the image sensor 130 and store them in thetemporary memory 115. Once the succession was complete or the temporarymemory was full, the image processor 120 would compress the capturedimages and store the compressed images on the memory card 110, perhapsin a predefined common format such as JPEG. Prior to the presentinvention, each image would be stored in a separate file, repeating allcommon information in each file as well as auxiliary information such asdate, time and resolution.

The present invention contemplates storing multiple images in one file,with perhaps one set of auxiliary information. Furthermore, instead ofstoring a series of discrete and complete images in the file, a baseimage is stored, and for each additional image, delta information isstored, thereby reducing or eliminating redundant data in the file thatis common to all images within the set. In one embodiment of the presentinvention, each image of the series is captured by the image processor120 and stored in the temporary memory 115. Once the succession iscomplete, the images from the temporary memory 115 are analyzed fordifferences between the base image and each of the subsequently capturedimages. The information regarding the base image is stored, along withinformation regarding any changes (delta) between the base image andeach of the subsequent images that have been captured, is stored inrelatively longer term memory, such as the memory card 110.

In another embodiment, which may utilize a fast image processor 120, thefirst image is written to the temporary memory 115 as it is captured,then as subsequent images are captured, only the delta information iswritten to the temporary memory 115. This embodiment has an additionaladvantage of reducing the amount of temporary memory consumed by eachimage of the succession, thereby permitting a greater number ofsequential images to be taken for a given amount of temporary memory115.

Referring now to FIG. 2, a flow chart of one aspect of the presentinvention is shown. Initially, an output file may be created forreceiving the image information (205). Next, a series of n images arecaptured (210). The images may be stored in the temporary memory duringprocessing. Once at least two images of the series of images arecaptured, the first image, possibly in compressed form, is written tothe output file (230). The output file may be an image file according tothe present invention that is stored on the memory card 110. Next, foreach subsequent image in the series of images (235), the current imageis compared to the previous image, or the base image, and differencesbetween the present image and the previous image or base image areidentified (240). The difference information is then written to orappended to the output file (250). If there are additional images in theseries (260), the prior two steps (240 and 250) are repeated for eachsubsequent image until the last image is processed. The output file maythen be closed (270).

Referring now to FIG. 3, a flow chart of a second aspect of the presentinvention is shown. Initially, a temporary file may be created (305).Next, a first image is captured (310). The first image is stored in thetemporary file 320. The first image may be stored in compressed form,possibly in a JPEG format. The temporary file is stored in the imagememory 115 to gain speed, since image memory 115 may be much faster thanmemory card 110. Next, for each subsequent image (335), the currentimage is compared to the previous image, and differences between thepresent image and the previous image or base image are identified (340).The difference information is then written or appended to the temporaryfile (350). If there are additional images in the series (360), theprior two steps (340 and 350) are repeated for each subsequent imageuntil the last image is processed. Once the series is complete, theoutput file is created 370 and the information in the temporary file iscopied to the output file 380. The movement of the information from thetemporary file to the output file may optionally include furthercompressing of the image information, and thus the information on theoutput file may or may not be a direct bit-for-bit copy of theinformation in the temporary file.

Referring now to FIG. 4, a set of three sample images is depicted thatmay be processed according to the present invention. In this example,the image 310 consists of a car 311 and a stop sign 312. The images 320and 330 also include a car (321 and 331) and a stop sign (322 and 332).The only difference between the three images (310, 320, and 330) is thelocation of the car (311, 321, and 331), in that the car is located indifferent positions in each image (310, 320, and 330). In the method ofthe present invention, the first image 310 captured would be recordedand stored, either on memory card 110 or in a temporary file in memory115. The first, or base, image may be stored in compressed format. Thesecond image 320 is captured, and it is compared to the first image,instead of storing the entire image as a separate image file. When thecomparison is made, it is determined that the background as well as thestop sign are the same, and in the same position, and that only the car321 is different (e.g., in position). Therefore, information regardingthe movement of object 311 to the position of object 321 may be recordedin the output file or temporary file, or information regarding thechanged sections, sub-picture or pixels may be written. When the thirdimage 330 is captured, it would be compared to the first image, or in apreferred embodiment, to the second image, and instead of storing theentire image as a separate image file, it is compared to the storedimage and it is determined that the entire background as well as thestop sign are the same and in the same position, and that only the car331 is different. Therefore, information regarding the movement ofobject 321 to the position of object 331 may be recorded in the outputfile or temporary file, or information regarding the changed sections,sub-picture or pixels may be written. Optionally, the movement of object311 to the position of object 331 could be recorded. Once all images inthe sequence (310, 320, and 330) are captured and compressed, if theimages have been written to the output file, the output file isfinalized and closed. If the images are being written to a temporaryfile, the temporary file is either copied to the memory card 110 or anoutput file is created on the memory card 110 and the contents of thetemporary file is transferred to the output file. Optionally, changesmay be made to the information as it is transferred, possibly forachieving further compression of the file.

It is believed that the system and method of the present invention andmany of its attendant advantages will be understood by the foregoingdescription. It is also believed that it will be apparent that variouschanges may be made in the form, construction and arrangement of thecomponents thereof without departing from the scope and spirit of theinvention or without sacrificing all of its material advantages. Theform herein before described being merely exemplary and explanatoryembodiment thereof. It is the intention of the following claims toencompass and include such changes.

1. A method of compressing a series of still digital images comprising:capturing a first digital image of a series of digital images; storingsaid first digital image in an output file; capturing at least one moredigital image of said series of digital images; and for each digitalimage of said at least one more digital image: comparing said eachdigital image to said first digital image to create a set of differencedata; appending said set of difference data to said output file.
 2. Themethod of compressing a series of still digital images of claim 1wherein said output file is located in a temporary memory.
 3. The methodof compressing a series of still digital images of claim 1 wherein saidoutput file is located in a memory card.
 4. The method of compressing aseries of still digital images of claim 1 wherein said comparing isperformed by comparing sub-picture elements.
 5. The method ofcompressing a series of still digital images of claim 1 furthercomprising: closing said output file.
 6. The method of compressing aseries of still digital images of claim 2 further comprising: closingsaid output file and copying said output file from said temporary memoryto a memory card.
 7. A method of compressing a series of still digitalimages comprising: capturing a first digital image of a series ofdigital images; storing said first digital image in an output file;capturing at least one more digital image of said series of digitalimages; and for each digital image of said at least one more digitalimage; comparing said each digital image to a previous digital image tocreate a set of difference data; appending said set of difference datato said output file.
 8. The method of compressing a series of stilldigital images of claim 7 wherein said output file is located in atemporary memory.
 9. The method of compressing a series of still digitalimages of claim 7 wherein said output file is located in a memory card.10. The method of compressing a series of still digital images of claim7 wherein said comparing is performed by comparing sub-picture elements.11. The method of compressing a series of still digital images of claim7 further comprising: closing said output file.
 12. The method ofcompressing a series of still digital images of claim 8 furthercomprising: closing said output file and copying said output file fromsaid temporary memory to a memory card.
 13. A camera comprising: animage sensor; an image processor connected to said image sensor; amemory connected to said image processor; means in communication withsaid image processor for capturing a series of still digital images fromsaid image sensor; and means for compressing said series of stilldigital images by recording a base image and difference information foreach subsequent digital image in said series of still digital images.14. The camera of claim 13 wherein said means for compressing creates anoutput file located in a temporary memory.
 15. The camera of claim 13wherein said means for compressing creates an output file on a memorycard.
 16. The camera of claim 13 wherein said means for compressingcomprises measuring differences between said each subsequent image andstoring said differences in said memory.
 17. The camera of claim 13wherein said means for capturing a series of still digital imagescomprises holding a shutter mechanism to signal said image processor tocontinue capturing images until said shutter mechanism is disengaged.18. The camera of claim 13 wherein said means for capturing a series ofstill digital images comprises setting a control parameter using akeypad so that said image processor captures a sequence of at least twodigital images when a shutter mechanism is engaged.
 19. A method ofdecompressing a series of still digital images comprising: reading afirst digital image from a series of digital images stored in a singlefile; storing said first digital image in a first output file; and foreach subsequent digital image of said at least one more digital imagesstored in a single file; combining difference information related tosaid each subsequent digital image from said series of digital imageswith said first digital image creating a next digital image; storingsaid next digital image in a next output file.
 20. The method ofdecompressing a series of still digital images of claim 19 wherein saidsingle file is located in a file on a hard disk.
 21. The method ofdecompressing a series of still digital images of claim 19 wherein saidfirst output file is stored in a hard disk.